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Background/context: 

Understanding how to provide young children with an early foundation in school readiness 
skills is becoming a primary goal of many states in order to decrease the high incidence of school 
failure and drop-out, particularly by children from low income homes (Bowman, Donovan, & 
Burns, 2001; NAEP, 2003). For young children, a quality education includes teachers being 
skilled in the use of instructional approaches that are sensitive to the child’ s developmental needs 
and expose them to experiences with language, emergent literacy, and math within a responsive 
environment that supports social-emotional development (Bowman et al, 2001, Duncan et ah, 
2007, National Early Literacy Panel, 2008). Thus, in finding solutions to better preparing 
children for school, attention needs to be given to training teachers in instructional practices that 
support children’s learning of these cognitive skills (Howes et ah, 1992; Kontos et al., 1997). 
However, access to effective professional development has not kept up with the increased 
acknowledgement of its significance. 

Purpose/objective/research question/focus of study: 

This study was designed to enhance early childhood teachers’ efforts to prepare young 
children for school. Thus, the first objective was to determine if teachers receiving 
comprehensive professional development (i.e., PD) would show gains in language and literacy 
instructional practices than teachers randomized to the control condition. Objective 2 was to 
determine if teachers in the second year of the intervention showed greater gains in their 
instructional practices than teachers receiving the program for one year and whether gains in 
child outcomes were greater if they had teachers with 2 years versus 1 year of the program. 
Finally, as the study design had Year 1 control teachers becoming intervention teachers in Year 
2, objective 3 was to determine if across the same randomly selected teachers with and without 
the intervention there were greater gains in both teacher practices and child outcomes in year 2. 
Professional development included an on-line PD course with mentoring, the use of a 
technology-driven progress monitoring tool, and a research based curriculum. 

Setting: 

A request for applications was opened for all communities across Texas interested in 
participating. Applicants had to meet the following criteria: 1) bring together a leadership 
committee that crossed three early childhood programs (i.e., HS, childcare, school district), 2) 
identify a lead agency to coordinate offices for program staff, and 3) agree to random assignment 
of 20 classrooms to “business as usual’’ vs. participation in the program with stratification across 
the three program classrooms. In addition, the community partnership had to agree to: 1) use a 
research-based curriculum, 2) use the PD model including the progress monitoring system, a 
facilitated on-line course, and in-classroom mentoring, & 3) participate in all required trainings 
to ensure effective implementation. Of 17 communities that applied, 1 1 met the above criteria 
and were accepted into the program. See Appendix for list of communities (Table 1). 
Population/Participants/Subjects: 

Classrooms. In Year I, the 1 1 sites recruited 20 classrooms for random assignment to control 
or intervention conditions. To avoid condition bias, only one classroom per building site was 
allowed and a request was that the 20 classrooms had representation from the three service 
delivery agencies. Participation across the entire study was seen for 213 classrooms; public 
schools (n = 90), childcare center (n = 64), and Head Start (n = 59). The evaluation sample was 
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composed of a large proportion of children from minority backgrounds; 67 % Hispanic 
American, 19% African American, 12% Caucasian, 1% Asian American, 1% other ethnicity. 

Intervention/Program/Practice: 

The online professional development program included eight program courses that covered 
language and early literacy instructional areas (see Table 2 for list of course titles). The online 
course included (a) small-group interactive learning facilitated by a trainer with and independent 
review, (b) extensive videotaped modeling and expert commentaries, (c) active engagement of 
learning with online assessments of knowledge, (d) opportunities for practicing specific skills 
within the small group coursework, and (e) teachers’ posting of experience with specific 
instructional activities in their own classroom with trainer review and feedback. 

In-classroom teacher mentoring. Mentoring included planning for 3 hours of daily cognitive 
readiness activities in planful, purposeful, but playful ways, classroom demonstration of how to 
implement activities effectively, and side by side coaching. 

The child progress monitoring system was implemented with the use of Personal Digital 
Assistant (PDA) technology. The system included evaluation of vocabulary and letter knowledge 
using a 60 second fluency method. The system also included individual child assessment of 
phonological awareness by asking the child to demonstrate knowledge of rhyming words, words 
in sentence segmentation, and onset rhyme. All classrooms in the intervention had a curriculum 
off the state approved list of pre-kindergarten curricula. See Table 3 for list of curricula. 
Research Design: 

As this was a state funded initiative, there were constraints on the design including the 
requirement that by the end of the second year all teachers involved in the program would have 
received the PD program. Additionally, due to funding delays, the first year of intervention was 
abbreviated. The first objective was to examine teacher change in instructional practices for 
control versus target teachers across 4.5 months. To address the first objective, teachers in each 
of the 1 1 communities (n = 20 classes per community) were randomly assigned to the TEEM 
intervention (i.e., Texas Early Education Model) condition versus a control condition. The 
second objective was to determine if increased exposure to PD resulted in greater change in 
teacher practices as well as child language and literacy skills. This was possible as in the second 
year, previously trained teachers received another year of the program and the previous control 
teachers were entered into their first year of the program. Thus, for year 2, the randomly 
assigned TEEM condition teachers received a second year of the intervention and the control 
teachers received TEEM for the first time. Einally, a third objective was to examine differences 
in instructional practices and child outcomes within a small group of the same randomly selected 
teachers without the program (Year 1) and with program participation (Year 2). 

Data Collection and Analysis: 

Evaluation of teacher outcomes. Assessment of change in instructional practices was based on 
classroom observations of 84 teachers divided between the control and target classrooms in Year 
1 (Objective 1). This same process was repeated in Year 2 with a re -randomization of teachers 
selected for observation (Objective 2). Because the random selection for observations was 
repeated each year, a smaller number of teachers (n = 14) had observations when they were 
control classrooms in Year 1 and in Year 2 when they received their first year of the program 
(Objective 3). The Teacher Behavior Rating Scale (TBRS, Landry, Crawford, Swank, & 
Gunnewig, 2002) was used to evaluate teaching practices. The TBRS subscales include: 1) 
responsive teaching, 2) centers, 3) lesson plans, 4) oral language, 5) book reading, 6) print & 
letter knowledge, 7) written expression, 8) phonological awareness, and 9) math. 
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Evaluation of child outcomes. To evaluate child outcomes, we randomly selected seven 
consented children for testing per class (Year 1, n = 717 controls and n = 710 from intervention 
classrooms). In Year 2,n = 802 children had first year teachers and n = 528 children with second 
year teachers. For bilingual classrooms, the teacher was interviewed using a systematic set of 
questions to determine language of assessment. Assessments of child outcomes included the 
Expressive One Word Picture Vocabulary Test (EOWPVT; Brownell, 2000) ; the Preschool 
Language Scale - 4* Ed. (PLS-4; Zimmerman, Steiner, & Pond, 2002) to assess complex 
receptive language development. The Developing Skills Checklist (DSC; CTB/ McGraw Hill, 
1990) Auditory subscale to examine phonological awareness development; The Preschool 
Comprehensive Test of Phonological and Print Processing (Pre-CTOPPP; Lonigan, Wagner, 
Torgesen, & Rashotte, 2003) to assess print awareness and letter knowledge. 

Analysis of covariance (i.e., ANCOVA) examined differences between 84 teachers randomly 
selected between control and target teachers on post -test TBRS scores, controlling for pre-test 
TBRS scores (Objective 1). ANCOVAs examined the extent to which more experience with 
TEEM made a difference in teaching behaviors between teachers (n = 84) who were in their first 
year vs. those in their second year, controlling for pre-test TBRS scores (Objective 2). Analysis 
of variance (i.e., ANOVA) examined post-test differences between teachers who were randomly 
selected in Year 1 for measurement with the TBRS as control teachers to those same teachers 
who were also randomly selected for observation as program teachers in Year 2 (n = 14) 
(Objective 3). 

Child skills. Child testing occurred during winter and spring of Year 1 and fall and spring of 
Year 2. The children in Year 1 vs. Year 2 classrooms were different and thus, the student 
observations were independent across years and nested within classrooms. Although we did not 
expect differences for child outcomes in Year 1 given the short time period, analyses confirmed 
these expectations (Objective 1). Eor Objective 2, ANCOVAs examined post-test child scores 
with the pretest, age at pre-test, and time between assessments used as covariates. Interactions 
among covariates and treatment conditions were retained if significant. Treatment condition (first 
vs. second year of program participation) was the independent variable. Eor Objective 3, the 
ANOVA analyses were conducted using posttest data only and control vs. program year was the 
independent variable. In addition to the independent variable in each analysis, language of 
testing and child’s age at posttest were included in models. To evaluate the significance of the 
results, effects sizes are reported as Cohen’s d (Cohen, 1988). 

Findings/Results: 

Objective 1: Effect of Program on Teaching Behaviors Significant differences on most of the 
TBRS subscales were found in favor of target teachers relative to control teachers. The results 
are summarized in Table 3 and illustrated in Eigure 1 . With just 4 months of the program, target 
teachers were observed to show greater gains in their use of language building activities and 
emergent literacy instructional practices including phonological processing and print knowledge 
activities with large effect sizes. In addition, teachers who received the PD program showed 
more responsive teaching practices, organization of their centers, and the use of lesson plans as 
well as early math instruction (effects were moderate to large). 

Objective 2: Effect of Length of Program Participation on Teaching Behaviors and Child 
Skills In Year 2, teachers who were in the control condition during year one moved into the 
program (first year program teachers) while teachers who were in the program during year one 
continued to receive more training (second year program teachers). No statistically significant 
differences on the TBRS subscales were observed between teachers in their second year of 
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program exposure versus teachers in their first year. However, obtained effect sizes would be 
considered small to moderate. 

Child language. The model for children’s vocabulary skills revealed a significant effect of the 
length of their teachers’ program participation that depended upon age at pretest, F(l, 1061) = 
4.73 ; p < .03, effect size = .16. In addition, the program effect was dependent upon the child’s 
language of testing, F(l, 1061 = 4.29; p < .04); effect size = .35. Children of teachers in their 
second program year scored higher on vocabulary at the post and this was especially true for 
those who were younger at the pretest (Figure 2). In addition, children tested in Spanish with 
teachers in their second year had higher posttest vocabulary scores. 

A significant program effect was found for children’s complex language, F(l, 1107) = 14.44, 
p = .0002. However, this depended on the pretest score and age at pretest, F(l, 1107) = 19.49; p 

< .0001, as well as the language of testing, F(l, 1 107) = \ \.A\,p = .0008 (Figure 2). Children 
tested in English had higher post -test scores regardless of the amount of teacher training but 
those tested in Spanish had higher posttest scores if their teachers were in the second year of 
training, effect size = .34. In addition, the program effect for a second year of teacher training 
was greater for younger children when they started out with higher scores at pretest, effect size = 
.24, but greater for older children when they started out with lower scores, effect size = .20. 

Comparison of child letter knowledge and print awareness. Significant differences were 
found for the print knowledge total score but this was dependent on their pretest score, F(l, 

1118) = 9.29, p < .003), effect size = .34 (Figure 3a). This included differences on knowledge of 
letter sounds, letter names, and print discrimination. The finding indicated that the posttest scores 
were significantly higher for children with teachers in their second program year, but this was 
particularly true for those children who showed lower scores at the pretest. 

Child phonological awareness. On the Developing Skills Checklist Auditory subtest (i.e., PA) 
there was a significant group by age by language of testing interaction, F(l, 1116) = 6.13;/)< 

.02 (Figure 3). Children tested in Spanish, particularly those who were older at the beginning of 
the year, had higher posttest scores if their teacher was in their second year of training, effect 
size = .50. Conversely, children of second year teachers who were tested in English seemed to do 
better if they were younger at the beginning of the year, effect size =.26 (Eigure 3b). 

Objective 3: Comparison of the Same Teachers without and with the Program Across the two 
years, these teachers had two sets of children that could be compared (control classroom, n = 

423; first year of program classroom, n = 467). Differences were compared on post-test TBRS 
data and post-test child data. Thus, while the TBRS analyses were based on 14 randomly 
selected teachers with data across both years, the child outcomes were based on all children 
tested in control classrooms for Year 1 and first year program classroom in Year 2. 

Comparison of teaching behaviors. The comparison of teaching behaviors for the same 
teachers as control vs. target teachers showed similar, but stronger effects than the teaching 
results for Objective 1 (Table 4, Eigure 4). Very large effect sizes were found for language 
building, book reading, and emergent literacy activities. Teachers who participated in the PD 
program demonstrated more responsive teaching, and best practices, for which there was a very 
large effect size. 

Comparison of child vocabulary. Results based on the EOWPVT indicated that the 
scores for the children who were taught by teachers after they participated in the program were 
significantly higher than scores for the children under the control condition, F(l, 904) = 17.90; p 

< .0001, and that this depended upon the age of the child, F( 1,904) = 5.63;/>< .02, effect size = 
.26 (Eigure 5). The greatest differences between the groups were found for older children. The 
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relation of test language to the scores on the EOWPVT also depended on the age of the child, 
F(l, 904) = 4.92; p < .03 with the greatest difference found for children tested in English as 
compared to Spanish, but only for older children. 

Comparison of child letter knowledge and print awareness. The final model for the total 
score on Print Awareness included the child’s age, F(l, 918) = 89.07 ; p < .0001, and program vs. 
control years, F(l, 918) = 4.93; p < .03, effect size .11, Eigure 5. The results indicated that the 
children tested while their teachers were in the control condition scored significantly lower than 
those tested when their teachers were participating in the program. The program effects were 
larger for naming letters, F(l, 916) = 14.3 1; p = .0002; effect size = .20. 

Conclusions: 

Results show evidence that these components are effective in promoting greater change in the 
teachers’ practices when compared to the control teachers (Objective 1). Greater gains with 
moderate to large effect sizes were found for the program teachers in the initial 4-month training 
period across most TBRS subscales. Some of the greatest gains were found in school readiness 
instructional activities. Although the amount of time a teacher participated in the program was 
expected to result in differences in instructional practices, no differences were found. 

Differences in the teachers’ exposure to the program were apparent when examining the 
children’s development of language and early literacy skills. While a positive effect of additional 
teacher participation was seen for all skills, at times these were dependent upon the child’s age 
and language of assessment. Eor example, there were stronger gains in language skills for 
children whose teachers were in their second year of the program. However, these were apparent 
for children who were younger at the beginning of the year. Language skill differences also were 
stronger for those children evaluated in Spanish. As children tested in Spanish would be learning 
English, they might be considered as children who needed the most support from teachers. 

The two aspects of emergent literacy that have been found to be unique predictors of reading 
from the recently released National Institute for Literacy report (2008), also showed greater 
increases if children’ s teachers were in their second year of the program. Eor letter-sound 
correspondence this finding was strongest for children who were younger while print 
discrimination skills were strongest for children learning English. The effect of amount of 
teacher participation also was seen for phonological awareness skills. However, unlike the other 
outcomes, those children who were older at the beginning of school and their teachers had more 
experienced with the program showed the greatest gains. This may occur because of the 
complexity of phonological awareness, such that a more experienced teacher is better able to 
implement these types of activities. 

Einally, the design of this evaluation allowed for comparison of a small group of control 
teachers in year 1 and again as year 2 program teachers. The differences in these teachers’ 
instructional practices before and after the program were found in almost all areas of teaching 
and the improvements were strong. The different groups of children in these teachers’ 
classrooms also showed differences in their language and emergent literacy gains across the 
school year. As the 1 1 community partnerships all had Head Start, public school, and child care 
classrooms, this study demonstrates that comprehensive professional development, mentoring, 
progress monitoring, and collaborative relationships between educators can result in benefits for 
at-risk children. The results can inform state education policy by demonstrating the need for a 
comprehensive approach to preparing teachers in early childhood classrooms in order for 
children from low income homes to show growth in language and early literacy skills. 



2009 SREE Conference Abstract Template 




Early childhood professional development - 7 



Appendixes 

Not included in page count. 

Appendix A. References 

Abrams & Co. (2003). Let’s begin with the letter people. Waterbury, CT: Abrams & Co., Pub. 

Bowman, B. T., Donovan, M. S. & Burns, M. S. (Eds.). (2001). Eager to learn: Educating our 
preschoolers. Washington, DC: National Academy Press. 

Brownell, R (2000). Expressive one-word picture vocabulary test. Novato, CA: Academic 
Therapy. 

Duncan, G.J., Dowsett, C.J., Claessens, A., Magnuson, K., Huston, A.C., Klebanov, P., et 
al. (2007). School readiness and later achievement. Developmental Psychology, 43, 
1428-1446. 

Howes, C., Phillips, D., & Whitebooks, M. (1992). Thresholds of quality: Implications for the 
social development of children in center based ehild care. Child Development, 63, 449-460. 

Kontos, S.C., Howes, C., & Galinsky, E. (1997). Does training make a difference to quality in 
family child care? Early Childhood Research Quarterly, 12, 351-372. 

Landry, S., Crawford, A., Gunnewig, S., & Swank, P., (2002). Teacher Behavior Rating Scale. 
Unpublished research instrument. Center for Improving the Readiness of Children for 
Learning and Education, University of Texas Health Science Center at Houston. 

Lonigan, C. J., Wagner, R. K., Torgesen, J. K., & Rashotte, C. A. (2003). Preschool 
comprehensive test of phonological and print processing. Austin, TX: PROED. 

National Assessment for Educational Progress. (2003). Highlighting NAEP 2003. Retrieved July 
17, 2007, from http://ww . ode.state.or.us/initiatives/naep/naepnews. 

National Early Literacy Panel (2008). Report of the National Early Literacy Panel. Washington 
DC: National Institute for Literacy. 

Rigby/Harcourt (2002). Pebble soup. Orlando, EL:Rigby/Harcourt. 

Scholastic, Inc. (2003). Building language and literacy. NY: Scholastic 

Sopris West (2003). We can! Longmont, CO: Sopris West. 

SRA/McGraw-Hill (2003). The DEM childhood express. Columbus, OH: SRA/McGraw-Hill. 
Wright Group/McGraw (2002). Doors to discovery. Bothell, WA. 

Zimmerman, I. L., Steiner, V. G. & Pond, R. E. (2002). Preschool Language Scale, (d*** ed). San 
Antonio, TX: The Psychological Corporation. 



2009 SREE Conference Abstract Template 



Early childhood professional development - 8 



Appendix B. Tables and Figures 
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Table 1. 

Community Partnerships at the Completion of the First Year 



Community 

Partnership 


Independent 
School 
District 
n = 90 


Head Start 
n = 59 


Child Care 
n = 64 


Amarillo 


14 


2 


4 


Austin 


10 


3 


6 


Brownsville 


9 


5 


5 


Dallas 


4 


3 


13 


El Paso 


6 


6 


8 


Ft. Worth 


8 


8 


4 


Houston 


7 


8 


5 


Laredo 


8 


7 


5 


Raymondville 


9 


6 


3 


San Antonio 


6 


8 


4 


Wichita Falls 


9 


3 


7 
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Table 2. 

Approve Early Childhood Curricula. 



Curriculum Name 


Publisher 


Building Language and Literacy 


Scholastic Inc., 2003 


Let’s Begin with the Letter People 


Abrams & Co., 2003 


DLM Childhood Express 


SRA/ McGraw-Hill, 2003 


Doors to Discovery 


Wright Group/McGraw-Hill, 2002 


Pebble Soup 


Rigby /Harcourt, 2002 


We Can! 


Sopris West, 2003 



Note: Each classroom was provided with a set of materials to organize the classroom and a 
School Readiness kit. 
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Table 3. 

On-line Professional Development Course List 

• Classroom Management 

• Setting the Stage for Children’ s Talk 

• Reading Aloud 

• Phonological Awareness 

• Letter Knowledge 

• Mathematics 

• Written Expression 

• Language Development 
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Table 4. 

Summary of Teacher Behavior Change for Objective 1 and 3. 







Objective 1 






Objective 3^ 






d.f 


F 


P 


d 


F 


P 


d 


Teaching Behaviors 


Total 


1,82 


19.74 


.0001 


.84 


32.60 


.0001 


1.71 


Language 


Book reading 


1,79 


9.27 


.003 


.57 


39.56 


.0001 


2.27 


Oral language 


1,82 


5.89 


.017 


.40 


57.61 


.0001 


2.04 


Emergent Literacy 


Phonolog Awareness 


1,82 


9.10 


.003 


.66 


27.98 


.0001 


2.45 


Print Use 


1,82 


25.16 


.0001 


1.03 


25.50 


.0002 


1.98 


Written Expression 


1,81 


3.36 


.071 


.39 


3.61 


.0793 


.55 


Math 


Total 


1,81 


8.63 


.004 


.65 




Ns 




Best Practices 


Total 


1,82 


9.78 


.002 


.56 


40.32 


.0001 


2.21 
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Classroom Organization 



Centers 


1,81 


8.48 


.005 


.63 


19.18 


.0007 


1.22 


Lesson plans 


1,79 


6.15 


.015 


.59 


19.36 


.0007 


1.45 



‘All J./. = 1, 13 
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Figure Captions 

Figure 1 . Comparison after 4.6 months of program implementation of teaching behaviors for 
control versus program teachers - Year 1 . 

Figure 2. Effects on children’s expressive vocabulary (top) and receptive language (bottom) 
skills. 

Figure 3. Effects on children’s on print awareness (top) and phonological awareness (bottom) 
skills. 

Figure 4. Comparison of teaching behaviors for the same group of teachers when in control 
classrooms (Year 1) versus first year of program (Year 2). 

Figure 5. Effects on vocabulary skills (top) and print awareness (bottom) for children with 
teachers in control classrooms (Year 1) versus in the first year of program (Year 2). 
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PLS Receptive Language Raw Score 
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DSC Phonological Awareness Total Raw Score 
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Print Awareness Raw Score EOWPVT Vocabularly Raw Score 
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